Method and apparatus for encoding and decoding image by using large transformation unit

ABSTRACT

Disclosed are an image encoding method and apparatus for encoding an image by grouping a plurality of adjacent prediction units into a transformation unit and transforming the plurality of adjacent prediction into a frequency domain, and an image decoding method and apparatus for decoding an image encoded by using the image encoding method and apparatus.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a continuation of U.S. application Ser. No.13/670,149, filed in the U.S. Patent and Trademark Office on Nov. 6,2012, which is a continuation of U.S. application Ser. No. 13/487,371,filed in the U.S. Patent and Trademark Office on Jun. 4, 2012, now U.S.Pat. No. 8,311,348, which is a continuation of Ser. No. 13/348,134,filed in the U.S. Patent and Trademark Office on Jan. 11, 2012, now U.S.Pat. No. 8,204,320, which is a continuation of U.S. application Ser. No.12/855,884, filed in the U.S. Patent and Trademark Office on Aug. 13,2010, which claims priority from Korean Patent Application No.10-2009-0074895, filed on Aug. 13, 2009, in the Korean IntellectualProperty Office, the disclosures of which are incorporated herein byreference in their entireties.

BACKGROUND

1. Field

The exemplary embodiments relate to a method and apparatus for encodingand decoding an image, and more particularly, to a method and apparatusfor encoding and decoding an image by transforming an image of a pixeldomain into coefficients of a frequency domain.

2. Description of the Related Art

In order to perform image compression, most of image encoding anddecoding methods and apparatuses encode an image by transforming animage of a pixel domain into coefficients of a frequency domain. Adiscrete cosine transform (DCT), which is one of frequency transformtechniques, is a well-known technique that is widely used in image orsound compression. An image encoding method using the DCT involvesperforming the DCT on an image of a pixel domain, generating discretecosine coefficients, quantizing the generated discrete cosinecoefficients, and performing entropy coding on the generated discretecosine coefficients.

SUMMARY

The exemplary embodiments provide a method and apparatus for encodingand decoding an image by using more efficient discrete cosine transform(DCT), and also provide a computer readable recording medium havingrecorded thereon a program for executing the method.

According to an aspect of an exemplary embodiment, there is provided animage encoding method including the operations of setting atransformation unit by selecting a plurality of adjacent predictionunits; and transforming the plurality of adjacent prediction units intoa frequency domain according to the transformation unit, and generatingfrequency component coefficients; quantizing the frequency componentcoefficients; and performing entropy encoding on the quantized frequencycomponent coefficients.

The operation of setting the transformation unit may be performed basedon a depth indicating a level of size-reduction that is graduallyperformed from a maximum coding unit of a current slice or a currentpicture to a sub-coding unit comprising the plurality of adjacentprediction units.

The operation of setting the transformation unit may be performed byselecting a plurality of adjacent prediction units on which predictionis performed according to a same prediction mode.

The same prediction mode may be an inter-prediction mode or anintra-prediction mode.

The image encoding method may further include the operation of settingan optimal transformation unit by repeatedly performing theaforementioned operations on different transformation units, wherein theaforementioned operations include the operations of setting thetransformation unit by selecting a plurality of adjacent predictionunits, transforming the plurality of adjacent prediction units into thefrequency domain according to the transformation unit and generating thefrequency component coefficients, quantizing the frequency componentcoefficients and performing the entropy encoding on the quantizedfrequency component coefficients.

According to another aspect of an exemplary embodiment, there isprovided an image encoding apparatus including a transformer for settinga transformation unit by selecting a plurality of adjacent predictionunits, transforming the plurality of adjacent prediction units into afrequency domain according to the transformation unit, and generatingfrequency component coefficients; a quantization unit for quantizing thefrequency component coefficients; and an entropy encoding unit forperforming entropy encoding on the quantized frequency componentcoefficients.

According to another aspect of an exemplary embodiment, there isprovided an image decoding method include the operations ofentropy-decoding frequency component coefficients that are generated bybeing transformed to a frequency domain according to a transformationunit; inverse-quantizing the frequency component coefficients; andinverse-transforming the frequency component coefficients into a pixeldomain, and reconstructing a plurality of adjacent prediction unitscomprised in the transformation unit.

According to another aspect of an exemplary embodiment, there isprovided an image decoding apparatus including an entropy decoder forentropy-decoding frequency component coefficients that are generated bybeing transformed to a frequency domain according to a transformationunit; an inverse-quantization unit for inverse-quantizing the frequencycomponent coefficients; and an inverse-transformer forinverse-transforming the frequency component coefficients into a pixeldomain, and reconstructing a plurality of adjacent prediction unitscomprised in the transformation unit.

According to another aspect of an exemplary embodiment, there isprovided a computer readable recording medium having recorded thereon aprogram for executing the image encoding and decoding methods.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features of the exemplary embodiments will becomemore apparent by describing in detail exemplary embodiments thereof withreference to the attached drawings in which:

FIG. 1 is a block diagram of an image encoding apparatus according to anexemplary embodiment;

FIG. 2 is a diagram of an image decoding apparatus according to anotherexemplary embodiment;

FIG. 3 is a diagram of a hierarchical coding unit according to anotherexemplary embodiment;

FIG. 4 is a block diagram of an image encoder based on a coding unitaccording to another exemplary embodiment;

FIG. 5 is a block diagram of an image decoder based on a coding unitaccording to another exemplary embodiment;

FIG. 6 illustrates a maximum coding unit, sub-coding units, andprediction units according to another exemplary embodiment;

FIG. 7 is a diagram of a coding unit and a transformation unit accordingto another exemplary embodiment;

FIGS. 8A and 8B illustrate division shapes of a maximum coding unit, aprediction unit, and a transformation unit according to anotherexemplary embodiment;

FIG. 9 is a block diagram of an image encoding apparatus according toanother exemplary embodiment;

FIG. 10 is a diagram of the transformer;

FIGS. 11A through 11C illustrate types of a transformation unitaccording to another exemplary embodiment;

FIG. 12 illustrates different transformation units according to anotherexemplary embodiment;

FIG. 13 is a block diagram of an image decoding apparatus according toanother exemplary embodiment; and

FIG. 14 is a flowchart of an image encoding method, according to anexemplary embodiment.

FIG. 15 is a flowchart of an image decoding method, according to anotherexemplary embodiment.

DETAILED DESCRIPTION

Hereinafter, the exemplary embodiments will be described in detail withreference to the attached drawings. In the exemplary embodiments, “unit”may or may not refer to a unit of size, depending on its context, and“image” may denote a still image for a video or a moving image, that is,the video itself.

FIG. 1 is a block diagram of an apparatus 100 for encoding an image,according to an exemplary embodiment.

Referring to FIG. 1, the apparatus 100 includes a maximum encoding unitdividing unit 110, an encoding depth determining unit 120, an image dataencoder 130, and an encoding information encoder 140.

The maximum encoding unit dividing unit 110 can divide a current pictureor slice based on a maximum coding unit that is an encoding unit of thelargest size. That is, the maximum encoding unit dividing unit 110 candivide the current picture or slice to obtain at least one maximumcoding unit.

According to an exemplary embodiment, an encoding unit can berepresented using a maximum coding unit and a depth. As described above,the maximum coding unit indicates an encoding unit having the largestsize from among coding units of the current picture, and the depthindicates the size of a sub coding unit obtained by hierarchicallydecreasing the coding unit. As a depth increases, a coding unit candecrease in size from a maximum coding unit to a minimum coding unit,wherein a depth of the maximum coding unit is defined as a minimum depthand a depth of the minimum coding unit is defined as a maximum depth.Since the size of an coding unit decreases from a maximum coding unit asa depth increases, a sub coding unit of a k^(th) depth can include aplurality of sub coding units of a (k+n)^(th) depth (k and n areintegers equal to or greater than 1).

According to an increase of the size of a picture to be encoded,encoding an image in a greater coding unit can result in a higher imagecompression ratio. However, if a greater coding unit is fixed, an imagecannot be efficiently encoded by taking into account the continuouslychanging image characteristics.

For example, when a smooth area such as the sea or sky is encoded, thegreater an coding unit is, the compression ratio can increase. However,when a complex area such as people or buildings is encoded, the smalleran coding unit is, the more a compression ratio can increase.

Accordingly, according to an exemplary embodiment, a different maximumimage coding unit and a different maximum depth are set for each pictureor slice. Since a maximum depth denotes the maximum number of times bywhich a coding unit can decrease, the size of each minimum coding unitincluded in a maximum image coding unit can be variably set according toa maximum depth.

The encoding depth determining unit 120 determines a maximum depth. Themaximum depth can be determined based on calculation of Rate-Distortion(R-D) cost. The maximum depth may be determined differently for eachpicture or slice or for each maximum coding unit. The determined maximumdepth is provided to the encoding information encoder 140, and imagedata according to maximum coding units is provided to the image dataencoder 130.

The maximum depth denotes a coding unit having the smallest size, whichcan be included in a maximum coding unit, i.e., a minimum coding unit.In other words, a maximum coding unit can be divided into sub codingunits having different sizes according to different depths. This isdescribed in detail later with reference to FIGS. 8A and 8B. Inaddition, the sub coding units having different sizes, which areincluded in the maximum coding unit, can be predicted or transformedbased on processing units having different sizes. In other words, theapparatus 100 can perform a plurality of processing operations for imageencoding based on processing units having various sizes and variousshapes. To encode image data, processing operations such as prediction,transformation, and entropy encoding are performed, wherein processingunits having the same size may be used for every operation or processingunits having different sizes may be used for every operation.

For example, the apparatus 100 can select a processing unit that isdifferent from a coding unit to predict the coding unit.

When the size of a coding unit is 2N×2N (where N is a positive integer),processing units for prediction may be 2N×2N, 2N×N, N×2N, and N×N. Inother words, motion prediction may be performed based on a processingunit having a shape whereby at least one of height and width of a codingunit is equally divided by two. Hereinafter, a processing unit, which isthe base of prediction, is defined as a ‘prediction unit’.

A prediction mode may be at least one of an intra mode, an inter mode,and a skip mode, and a specific prediction mode can be performed foronly a prediction unit having a specific size or shape. For example, theintra mode can be performed for only prediction units having the sizesof 2N×2N and N×N of which the shape is a square. Further, the skip modecan be performed for only a prediction unit having the size of 2N×2N. Ifa plurality of prediction units exist in a coding unit, the predictionmode with the least encoding errors can be selected after performingprediction for every prediction unit.

Alternatively, the apparatus 100 can perform frequency transformation onimage data based on a processing unit having a different size from acoding unit. For the frequency transformation in the coding unit, thefrequency transformation can be performed based on a processing unithaving a size equal to or smaller than that of the coding unit.Hereinafter, a processing unit, which is the base of frequencytransformation, is defined as a ‘transformation unit’. The frequencytransformation may be a Discrete Cosine Transform (DCT) or a KarhunenLoeve Transform (KLT).

The encoding depth determining unit 120 can determine sub coding unitsincluded in a maximum coding unit using RD optimization based on aLagrangian multiplier. In other words, the encoding depth determiningunit 120 can determine the shapes of a plurality of sub coding unitsdivided from the maximum coding unit, wherein the plurality of subcoding units have different sizes according to their depths. The imagedata encoder 130 outputs a bitstream by encoding the maximum coding unitbased on the division shapes, i.e., the shapes which divide the maximumcoding unit, as determined by the encoding depth determining unit 120.

The encoding information encoder 140 encodes information about anencoding mode of the maximum coding unit determined by the encodingdepth determining unit 120. In other words, the encoding informationencoder 140 outputs a bitstream by encoding information about a divisionshape of the maximum coding unit, information about the maximum depth,and information about an encoding mode of a sub coding unit for eachdepth. The information about the encoding mode of the sub coding unitcan include information about a prediction unit of the sub coding unit,information about a prediction mode for each prediction unit, andinformation about a transformation unit of the sub coding unit.

Since sub coding units having different sizes exist for each maximumcoding unit and information about an encoding mode must be determinedfor each sub coding unit, information about at least one encoding modecan be determined for one maximum coding unit.

The apparatus 100 can generate sub coding units by equally dividing bothheight and width of a maximum coding unit by two according to anincrease of depth. That is, when the size of a coding unit of a k^(th)depth is 2N×2N, the size of a coding unit of a (k+1)^(th) depth is N×N.

Accordingly, the apparatus 100 according to an exemplary embodiment candetermine an optimal division shape for each maximum coding unit basedon sizes of maximum coding units and a maximum depth in consideration ofimage characteristics. By variably adjusting the size of a maximumcoding unit in consideration of image characteristics and encoding animage through the division of a maximum coding unit into sub codingunits of different depths, images having various resolutions can be moreefficiently encoded.

FIG. 2 is a block diagram of an apparatus 200 for decoding an imageaccording to an exemplary embodiment.

Referring to FIG. 2, the apparatus 200 includes an image data obtainingunit 210, an encoding information extracting unit 220, and an image datadecoder 230.

The image data obtaining unit 210 acquires image data according tomaximum coding units by parsing a bitstream received by the apparatus200 and outputs the image data to the image data decoder 230. The imagedata obtaining unit 210 can extract information about a maximum codingunit of a current picture or slice from a header of the current pictureor slice. In other words, the image data obtaining unit 210 divides thebitstream in the maximum coding unit so that the image data decoder 230can decode the image data according to maximum coding units.

The encoding information extracting unit 220 extracts information abouta maximum coding unit, a maximum depth, a division shape of the maximumcoding unit, an encoding mode of sub coding units from the header of thecurrent picture by parsing the bitstream received by the apparatus 200.The information about a division shape and the information about anencoding mode are provided to the image data decoder 230.

The information about a division shape of the maximum coding unit caninclude information about sub coding units having different sizesaccording to depths included in the maximum coding unit, and theinformation about an encoding mode can include information about aprediction unit according to sub coding unit, information about aprediction mode, and information about a transformation units.

The image data decoder 230 restores the current picture by decodingimage data of every maximum coding unit based on the informationextracted by the encoding information extracting unit 220. The imagedata decoder 230 can decode sub coding units included in a maximumcoding unit based on the information about a division shape of themaximum coding unit. A decoding process can include a prediction processincluding intra prediction and motion compensation and an inversetransformation process.

The image data decoder 230 can perform intra prediction or interprediction based on information about a prediction unit and informationabout a prediction mode in order to predict a prediction unit. The imagedata decoder 230 can also perform inverse transformation for each subcoding unit based on information about a transformation unit of a subcoding unit.

FIG. 3 illustrates hierarchical coding units according to an exemplaryembodiment.

Referring to FIG. 3, the hierarchical coding units according to anexemplary embodiment can include coding units whose width×heights are64×64, 32×32, 16×16, 8×8, and 4×4. Besides these coding units havingperfect square shapes, coding units whose width×heights are 64×32,32×64, 32×16, 16×32, 16×8, 8×16, 8×4, and 4×8 may also exist.

Referring to FIG. 3, for image data 310 whose resolution is 1920×1080,the size of a maximum coding unit is set to 64×64, and a maximum depthis set to 2.

For image data 320 whose resolution is 1920×1080, the size of a maximumcoding unit is set to 64×64, and a maximum depth is set to 4. For imagedata 330 whose resolution is 352×288, the size of a maximum coding unitis set to 16×16, and a maximum depth is set to 1.

When the resolution is high or the amount of data is great, it ispreferable, but not necessary, that a maximum size of a coding unit isrelatively great to increase a compression ratio and exactly reflectimage characteristics. Accordingly, for the image data 310 and 320having higher resolution than the image data 330, 64×64 can be selectedas the size of a maximum coding unit.

A maximum depth indicates the total number of layers in the hierarchicalcoding units. Since the maximum depth of the image data 310 is 2, acoding unit 315 of the image data 310 can include a maximum coding unitwhose longer axis size is 64 and sub coding units whose longer axissizes are 32 and 16, according to an increase of a depth.

On the other hand, since the maximum depth of the image data 330 is 1, acoding unit 335 of the image data 330 can include a maximum coding unitwhose longer axis size is 16 and coding units whose longer axis sizes is8, according to an increase of a depth.

However, since the maximum depth of the image data 320 is 4, a codingunit 325 of the image data 320 can include a maximum coding unit whoselonger axis size is 64 and sub coding units whose longer axis sizes are32, 16, 8 and 4 according to an increase of a depth. Since an image isencoded based on a smaller sub coding unit as a depth increases, theexemplary embodiment is suitable for encoding an image including moreminute details in scenes.

FIG. 4 is a block diagram of an image encoder 400 based on a codingunit, according to an exemplary embodiment.

An intra predictor 410 performs intra prediction on prediction units ofthe intra mode in a current frame 405, and a motion estimation unit 420and a motion compensation unit 425 perform inter prediction and motioncompensation on prediction units of the inter mode using the currentframe 405 and a reference frame 495.

Residual values are generated based on the prediction units output fromthe intra predictor 410, the motion estimation unit 420, and the motioncompensation unit 425, and the generated residual values are output asquantized transform coefficients by passing through a transformer 430and a quantization unit 440.

The quantized transform coefficients are restored to residual values bypassing through an inverse-quantization unit 460 and a frequencyinverse-transformer 470, and the restored residual values arepost-processed by passing through a deblocking unit 480 and a loopfiltering unit 490 and output as the reference frame 495. The quantizedtransform coefficients can be output as a bitstream 455 by passingthrough an entropy encoder 450.

To perform encoding based on an encoding method according to anexemplary embodiment, components of the image encoder 400, i.e., theintra predictor 410, the motion estimation unit 420, the motioncompensation unit 425, the transformer 430, the quantization unit 440,the entropy encoder 450, the inverse-quantization unit 460, thefrequency inverse-transformer 470, the deblocking unit 480 and the loopfiltering unit 490, perform image encoding processes based on a maximumcoding unit, a sub coding unit according to depths, a prediction unit,and a transformation unit.

FIG. 5 is a block diagram of an image decoder 500 based on a codingunit, according to an exemplary embodiment.

A bitstream 505 passes through a parsing unit 510 so that encoded imagedata to be decoded and encoding information necessary for decoding areparsed. The encoded image data is output as inverse-quantized data bypassing through an entropy decoder 520 and an inverse-quantization unit530 and restored to residual values by passing through a frequencyinverse-transformer 540. The residual values are restored according tocoding units by being added to an intra prediction result of an intrapredictor 550 or a motion compensation result of a motion compensationunit 560. The restored coding units are used for prediction of nextcoding units or a next picture by passing through a deblocking unit 570and a loop filtering unit 580.

To perform decoding based on a decoding method according to an exemplaryembodiment, components of the image decoder 500, i.e., the parsing unit510, the entropy decoder 520, the inverse-quantization unit 530, thefrequency inverse-transformer 540, the intra predictor 550, the motioncompensation unit 560, the deblocking unit 570 and the loop filteringunit 580, perform image decoding processes based on a maximum codingunit, a sub coding unit according to depths, a prediction unit, and atransformation unit.

In particular, the intra predictor 550 and the motion compensation unit560 determine a prediction unit and a prediction mode in a sub codingunit by considering a maximum coding unit and a depth, and the frequencyinverse-transformer 540 performs inverse transformation by consideringthe size of a transformation unit.

FIG. 6 illustrates a maximum coding unit, a sub coding unit, and aprediction unit, according to an exemplary embodiment.

The apparatus 100 and the apparatus 200 according to an exemplaryembodiment use hierarchical coding units to perform encoding anddecoding in consideration of image characteristics. A maximum codingunit and a maximum depth can be adaptively set according to the imagecharacteristics or variably set according to requirements of a user.

A hierarchical coding unit structure 600 according to an exemplaryembodiment illustrates a maximum coding unit 610 whose height and widthare 64 and maximum depth is 4. A depth increases along a vertical axisof the hierarchical coding unit structure 600, and as a depth increases,heights and widths of sub coding units 620 to 650 decrease. Predictionunits of the maximum coding unit 610 and the sub coding units 620 to 650are shown along a horizontal axis of the hierarchical coding unitstructure 600.

The maximum coding unit 610 has a depth of 0 and the size of a codingunit, i.e., height and width, of 64×64. A depth increases along thevertical axis, and there exist a sub coding unit 620 whose size is 32×32and depth is 1, a sub coding unit 630 whose size is 16×16 and depth is2, a sub coding unit 640 whose size is 8×8 and depth is 3, and a subcoding unit 650 whose size is 4×4 and depth is 4. The sub coding unit650 whose size is 4×4 and depth is 4 is a minimum coding unit, and theminimum coding unit may be divided into prediction units, each of whichis less than the minimum coding unit.

Referring to FIG. 6, examples of a prediction unit are shown along thehorizontal axis according to each depth. That is, a prediction unit ofthe maximum coding unit 610 whose depth is 0 may be a prediction unitwhose size is equal to the coding unit 610, i.e., 64×64, or a predictionunit 612 whose size is 64×32, a prediction unit 614 whose size is 32×64,or a prediction unit 616 whose size is 32×32, which has a size smallerthan the coding unit 610 whose size is 64×64.

A prediction unit of the coding unit 620 whose depth is 1 and size is32×32 may be a prediction unit whose size is equal to the coding unit620, i.e., 32×32, or a prediction unit 622 whose size is 32×16, aprediction unit 624 whose size is 16×32, or a prediction unit 626 whosesize is 16×16, which has a size smaller than the coding unit 620 whosesize is 32×32.

A prediction unit of the coding unit 630 whose depth is 2 and size is16×16 may be a prediction unit whose size is equal to the coding unit630, i.e., 16×16, or a prediction unit 632 whose size is 16×8, aprediction unit 634 whose size is 8×16, or a prediction unit 636 whosesize is 8×8, which has a size smaller than the coding unit 630 whosesize is 16×16.

A prediction unit of the coding unit 640 whose depth is 3 and size is8×8 may be a prediction unit whose size is equal to the coding unit 640,i.e., 8×8, or a prediction unit 642 whose size is 8×4, a prediction unit644 whose size is 4×8, or a prediction unit 646 whose size is 4×4, whichhas a size smaller than the coding unit 640 whose size is 8×8.

Finally, the coding unit 650 whose depth is 4 and size is 4×4 is aminimum coding unit and a coding unit of a maximum depth, and aprediction unit of the coding unit 650 may be a prediction unit 650whose size is 4×4, a prediction unit 652 having a size of 4×2, aprediction unit 654 having a size of 2×4, or a prediction unit 656having a size of 2×2.

FIG. 7 illustrates a coding unit and a transformation unit, according toan exemplary embodiment.

The apparatus 100 and the apparatus 200, according to an exemplaryembodiment, perform encoding with a maximum coding unit itself or withsub coding units, which are equal to or smaller than the maximum codingunit, divided from the maximum coding unit.

In the encoding process, the size of a transformation unit for frequencytransformation is selected to be no larger than that of a correspondingcoding unit. For example, when a encoding unit 710 has the size of64×64, frequency transformation can be performed using a transformationunit 720 having the size of 32×32.

FIGS. 8A and 8B illustrate division shapes of a coding unit, aprediction unit, and a transformation unit, according to an exemplaryembodiment.

FIG. 8A illustrates a coding unit and a prediction unit, according to anexemplary embodiment.

A left side of FIG. 8A shows a division shape selected by the apparatus100, according to an exemplary embodiment, in order to encode a maximumcoding unit 810. The apparatus 100 divides the maximum coding unit 810into various shapes, performs encoding, and selects an optimal divisionshape by comparing encoding results of various division shapes with eachother based on R-D cost. When it is optimal to encode the maximum codingunit 810 as it is, the maximum coding unit 810 may be encoded withoutdividing the maximum coding unit 810 as illustrated in FIGS. 8A and 8B.

Referring to the left side of FIG. 8A, the maximum coding unit 810 whosedepth is 0 is encoded by dividing it into sub coding units whose depthsare equal to or greater than 1. That is, the maximum coding unit 810 isdivided into 4 sub coding units whose depths are 1, and all or some ofthe sub coding units whose depths are 1 are divided into sub codingunits whose depths are 2.

A sub coding unit located in an upper-right side and a sub coding unitlocated in a lower-left side among the sub coding units whose depths are1 are divided into sub coding units whose depths are equal to or greaterthan 2. Some of the sub coding units whose depths are equal to orgreater than 2 may be divided into sub coding units whose depths areequal to or greater than 3.

The right side of FIG. 8A shows a division shape of a prediction unitfor the maximum coding unit 810.

Referring to the right side of FIG. 8A, a prediction unit 860 for themaximum coding unit 810 can be divided differently from the maximumcoding unit 810. In other words, a prediction unit for each of subcoding units can be smaller than a corresponding sub coding unit.

For example, a prediction unit for a sub coding unit 854 located in alower-right side among the sub coding units whose depths are 1 can besmaller than the sub coding unit 854. In addition, prediction units forsome (814, 816, 850, and 852) of sub coding units 814, 816, 818, 828,850, and 852 whose depths are 2 can be smaller than the sub coding units814, 816, 850, and 852, respectively. In addition, prediction units forsub coding units 822, 832, and 848 whose depths are 3 can be smallerthan the sub coding units 822, 832, and 848, respectively. Theprediction units may have a shape whereby respective sub coding unitsare equally divided by two in a direction of height or width or have ashape whereby respective sub coding units are equally divided by four indirections of height and width.

FIG. 8B illustrates a prediction unit and a transformation unit,according to an exemplary embodiment.

A left side of FIG. 8B shows a division shape of a prediction unit forthe maximum coding unit 810 shown in the right side of FIG. 8A, and aright side of FIG. 8B shows a division shape of a transformation unit ofthe maximum coding unit 810.

Referring to the right side of FIG. 8B, a division shape of atransformation unit 870 can be set differently from the prediction unit860.

For example, even though a prediction unit for the coding unit 854 whosedepth is 1 is selected with a shape whereby the height of the codingunit 854 is equally divided by two, a transformation unit can beselected with the same size as the coding unit 854. Likewise, eventhough prediction units for coding units 814 and 850 whose depths are 2are selected with a shape whereby the height of each of the coding units814 and 850 is equally divided by two, a transformation unit can beselected with the same size as the original size of each of the codingunits 814 and 850.

A transformation unit may be selected with a smaller size than aprediction unit. For example, when a prediction unit for the coding unit852 whose depth is 2 is selected with a shape whereby the width of thecoding unit 852 is equally divided by two, a transformation unit can beselected with a shape whereby the coding unit 852 is equally divided byfour in directions of height and width, which has a smaller size thanthe shape of the prediction unit.

FIG. 9 is a block diagram of an image encoding apparatus 900 accordingto another exemplary embodiment.

Referring to FIG. 9, the image encoding apparatus 900 according to thepresent exemplary embodiment includes a transformer 910, a quantizationunit 920, and an entropy encoder 930.

The transformer 910 receives an image processing unit of a pixel domain,and transforms the image processing unit into a frequency domain. Thetransformer 910 receives a plurality of prediction units includingresidual values generated due to intra-prediction or inter-prediction,and transforms the prediction units into a frequency domain. As a resultof the transform to the frequency domain, coefficients of frequencycomponents are generated. According to the present exemplary embodiment,the transform to the frequency domain may occur via a discrete cosinetransform (DCT) or Karhunen Loeve Transform (KLT), and as a result ofthe DCT or KLT, coefficients of frequency domain are generated.Hereinafter, the transform to the frequency domain may be the DCT,however, it is obvious to one of ordinary skill in the art that thetransform to the frequency domain may be any transform involvingtransformation of an image of a pixel domain into a frequency domain.

Also, according to the present exemplary embodiment, the transformer 910sets a transformation unit by grouping a plurality of prediction units,and performs the transformation according to the transformation unit.This process will be described in detail with reference to FIGS. 10,11A, 11B, and 12.

FIG. 10 is a diagram of the transformer 910.

Referring to FIG. 10, the transformer 910 includes a selection unit 1010and a transform performing unit 1020.

The selection unit 1010 sets a transformation unit by selecting aplurality of adjacent prediction units.

An image encoding apparatus according to the related art performsintra-prediction or inter-prediction according to a block having apredetermined size, i.e., according to a prediction unit, and performsthe DCT based on a size that is less than or equal to that of theprediction unit. In other words, the image encoding apparatus accordingto the related art performs the DCT by using transformation units thatare less than or equal to the prediction unit.

However, due to a plurality of pieces of header information added to thetransformation units, added overheads are increased as thetransformation units are decreased, such that a compression rate of animage encoding operation deteriorates. In order to solve this problem,the image encoding apparatus 900 according to the present exemplaryembodiment groups a plurality of adjacent prediction units into atransformation unit, and performs transformation according to thetransformation unit that is generated by the grouping. There is a highpossibility that the adjacent prediction units may include similarresidual values, so that, if the adjacent prediction units are groupedinto one transformation unit and then the transformation is performedthereon, a compression rate of an encoding operation may be highlyincreased.

For this increase, the selection unit 1010 selects the adjacentprediction units to be grouped into one transformation unit. Thisprocess will be described in detail with reference to FIGS. 11A through11C and 12.

FIGS. 11A through 11C illustrate types of a transformation unitaccording to another exemplary embodiment.

Referring to FIGS. 11A through 11C, a prediction unit 1120 with respectto a coding unit 1110 may have a division shape obtained by halving awidth of the coding unit 1110. The coding unit 1110 may be a maximumcoding unit, or may be a sub-coding unit having a smaller size than themaximum coding unit.

As illustrated in FIG. 11A, a size of the transformation unit 1130 maybe less than the prediction unit 1120, or as illustrated in FIG. 11B, asize of the transformation unit 1140 may be equal to the prediction unit1120. Also, as illustrated in FIG. 11C, a size of the transformationunit 1150 may be greater than the prediction unit 1120. That is, thetransformation units 1130 through 1150 may be set while having noconnection with the prediction unit 1120.

Also, FIG. 11C illustrates an example in which the prediction unit 1120is set by grouping a plurality of the prediction units 1120 included inthe coding unit 1110. However, a transformation unit may be set to begreater than a coding unit in a manner that a plurality of predictionunits, which are included not in one coding unit but in a plurality ofcoding units, are set as one transformation unit. In other words, asdescribed with reference to FIGS. 11A through 11C, a transformation unitmay be set to be equal to or less than a size of a coding unit, or to begreater than the size of the coding unit. That is, the transformationunit may be set while having no connection with the prediction unit andthe coding unit.

Although FIGS. 11A through 11C illustrate examples in which thetransformation unit has a square form. However, according to a method ofgrouping adjacent prediction units, the transformation unit may have arectangular form. For example, in a case where the prediction unit isnot set to have rectangular forms as illustrated in FIGS. 11A through11C but is set to have four square forms obtained by quadrisecting thecoding unit 1110, upper and lower prediction units, or left and rightprediction units are grouped so that the transformation unit may have arectangular form whose horizontal side or vertical side is long.

Referring back to FIG. 10, there is no limit in a criterion by which theselection unit 1010 selects the adjacent prediction units. However,according to the exemplary embodiment, the selection unit 1010 mayselect the transformation unit according to a depth. As described above,the depth indicates a level of size-reduction that is graduallyperformed from a maximum coding unit of a current slice or a currentpicture to a sub-coding unit. As described above with reference to FIGS.3 and 6, as the depth is increased, a size of a sub-coding unit isdecreased, and thus a prediction unit included in the sub-coding unit isalso decreased. In this case, if the transformation is performedaccording to a transformation unit that is less than or equal to theprediction unit, a compression rate of an image encoding operationdeteriorates since header information is added to every transformationunit.

Thus, with respect to a sub-coding unit at a depth of a predeterminedvalue, it is preferable, but not necessary, that prediction unitsincluded in the sub-coding unit are grouped and set as a transformationunit, and then the transformation is performed thereon. For this, theselection unit 1010 sets the transformation unit based on the depth ofthe sub-coding unit. For example, in the case where a depth of thecoding unit 1110 in FIG. 11C is greater than k, the selection unit 1010groups prediction units 1120 and sets them as a transformation unit1150.

Also, according to another exemplary embodiment, the selection unit 1010may group a plurality of adjacent prediction units on which predictionis performed according to the same prediction mode, and may set them asone transformation unit. The selection unit 1010 groups the adjacentprediction units on which prediction is performed according tointra-prediction or inter-prediction, and then sets them as onetransformation unit. Since there is a high possibility that the adjacentprediction units on which prediction is performed according to the sameprediction mode include similar residual values, it is possible to groupthe adjacent prediction units into the transformation unit and then toperform the transformation on the adjacent prediction units.

When the selection unit 1010 sets the transformation unit, the transformperforming unit 1020 transforms the adjacent prediction units into afrequency domain, according to the transformation unit. The transformperforming unit 1020 performs the DCT on the adjacent prediction unitsaccording to the transformation unit, and generates discrete cosinecoefficients.

Referring back to FIG. 9, the quantization unit 920 quantizes frequencycomponent coefficients generated by the transformer 910, e.g., thediscrete cosine coefficients. The quantization unit 920 may quantize thediscrete cosine coefficients that are input according to a predeterminedquantization step.

The entropy encoder 930 performs entropy encoding on the frequencycomponent coefficients that are quantized by the quantization unit 920.The entropy encoder 930 may perform the entropy encoding on the discretecosine coefficients by using context-adaptive variable arithmetic coding(CABAC) or context-adaptive variable length coding (CAVLC).

The image encoding apparatus 900 may determine an optimal transformationunit by repeatedly performing the DCT, the quantization, and the entropyencoding on different transformation units. A procedure for selectingthe adjacent prediction units may be repeated to determine the optimaltransformation unit. The optimal transformation unit may be determinedin consideration of an RD cost calculation, and this will be describedin detail with reference to FIG. 12.

FIG. 12 illustrates different transformation units according to anotherexemplary embodiment.

Referring to FIG. 12, the image encoding apparatus 900 repeatedlyperforms an encoding operation on the different transformation units.

As illustrated in FIG. 12, a coding unit 1210 may be predicted andencoded based on a prediction unit 1220 having a smaller size than thecoding unit 1210. A transformation is performed on residual values thatare generated by a result of the prediction, and here, as illustrated inFIG. 12, the DCT may be performed on the residual values based on thedifferent transformation units.

A first-illustrated transformation unit 1230 has the same size as thecoding unit 1210, and has a size obtained by grouping all predictionunits included in the coding unit 1210.

A second-illustrated transformation unit 1240 has sizes obtained byhalving a width of the coding unit 1210, and the sizes are obtained bygrouping every two prediction units adjacent to each other in a verticaldirection, respectively.

A third-illustrated transformation unit 1250 has sizes obtained byhalving a height of the coding unit 1210, and the sizes are obtained bygrouping every two prediction units adjacent to each other in ahorizontal direction, respectively.

A fourth-illustrated transformation unit 1260 is used when thetransformation is performed based on the fourth-illustratedtransformation unit 1260 having the same size as the prediction unit1220.

FIG. 13 is a block diagram of an image decoding apparatus 1300 accordingto another exemplary embodiment.

Referring to FIG. 13, the image decoding apparatus 1300 according to thepresent exemplary embodiment includes an entropy decoder 1310, aninverse-quantization unit 1320, and an inverse-transformer 1330.

The entropy decoder 1310 performs entropy decoding on frequencycomponent coefficients with respect to a predetermined transformationunit. As described above with reference to FIGS. 11A through 11C and 12,the predetermined transformation unit may be a transformation unitgenerated by grouping a plurality of adjacent prediction units.

As described above with reference to the image encoding apparatus 900,the transformation unit may be generated by grouping the adjacentprediction units according to a depth, or may be generated by grouping aplurality of adjacent prediction units on which prediction is performedaccording to the same prediction mode, that is, according to anintra-prediction mode or an inter-prediction mode.

The plurality of prediction units may not be included in one coding unitbut included in a plurality of coding units. In other words, asdescribed above with reference to FIGS. 11A through 11C, thetransformation unit that is entropy-decoded by the entropy decoder 1310may be set to be equal to or less than a size of a coding unit, or to begreater than the size of the coding unit.

Also, as described above with reference to FIG. 12, the transformationunit may be an optimal transformation unit selected by repeating aprocedure for grouping a plurality of adjacent prediction units, and byrepeatedly performing a transformation, quantization, and entropydecoding on different transformation units.

The inverse-quantization unit 1320 inverse-quantizes the frequencycomponent coefficients that are entropy-decoded by the entropy decoder1310.

The inverse-quantization unit 1320 inverse-quantizes the entropy-decodedfrequency component coefficients according to a quantization step thatis used in encoding of the transformation unit.

The inverse-transformer 1330 inverse-transforms the inverse-quantizedfrequency component coefficients into a pixel domain. Theinverse-transformer may perform an inverse-DCT on inverse-quantizeddiscrete cosine coefficients (i.e., the inverse-quantized frequencycomponent coefficients), and then may reconstruct a transformation unitof the pixel domain. The reconstructed transformation unit may includeadjacent prediction units.

FIG. 14 is a flowchart of an image encoding method, according to anexemplary embodiment.

Referring to FIG. 14, in operation 1410, an image encoding apparatussets a transformation unit by selecting a plurality of adjacentprediction units. The image encoding apparatus may select a plurality ofadjacent prediction units according to a depth, or may select aplurality of adjacent prediction units on which prediction is performedaccording to the same prediction mode.

In operation 1420, the image encoding apparatus transforms the adjacentprediction units into a frequency domain according to the transformationunit set in operation 1420. The image encoding apparatus groups theadjacent prediction units, performs a DCT on the adjacent predictionunits, and thus generates discrete cosine coefficients.

In operation 1430, the image encoding apparatus quantizes frequencycomponent coefficients, generated in operation 1420, according to aquantization step.

In operation 1440, the image encoding apparatus performs entropyencoding on the frequency component coefficients quantized in operation1430. The image encoding apparatus performs the entropy encoding on thediscrete cosine coefficients by using CABAC or CAVLC.

An image encoding method according to another exemplary embodiment mayfurther include an operation of setting an optimal transformation unitby repeatedly performing operations 1410 through 1440 on differenttransformation units. That is, by repeatedly performing thetransformation, the quantization, and the entropy encoding on differenttransformation units as illustrated in FIG. 12, it is possible to setthe optimal transformation unit.

FIG. 15 is a flowchart of an image decoding method, according to anotherexemplary embodiment.

Referring to FIG. 15, in operation 1510, an image decoding apparatusperforms entropy decoding on frequency component coefficients withrespect to a predetermined transformation unit. The frequency componentcoefficients may be discrete cosine coefficients.

In operation 1520, the image decoding apparatus inverse-quantizes thefrequency component coefficients that are entropy-decoded in operation1510. The image decoding apparatus inverse-quantizes the discrete cosinecoefficients by using a quantization step used in an encoding operation.

In operation 1530, the image decoding apparatus inverse-transforms thefrequency component coefficients, which have been inverse-quantized inoperation 1520, into a pixel domain and then reconstructs thetransformation unit. The reconstructed transformation unit is set bygrouping a plurality of adjacent prediction units. As described above,the transformation unit may be set by grouping the adjacent predictionunits according to a depth, or may be set by grouping a plurality ofadjacent prediction units on which prediction is performed according tothe same prediction mode.

According to the one or more exemplary embodiments, it is possible toset the transformation unit so as to be greater than the predictionunit, and to perform the DCT, so that an image may be efficientlycompressed and encoded.

The exemplary embodiments can also be embodied as computer-readablecodes on a computer-readable recording medium. The computer-readablerecording medium is any data storage device that can store data, whichcan be thereafter read by a computer system. Examples of thecomputer-readable recording medium include read-only memory (ROM),random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, andoptical data storage devices. The computer-readable recording medium canalso be distributed over network-coupled computer systems so that thecomputer-readable code is stored and executed in a distributed fashion.

For example, each of the image encoding apparatus, the image decodingapparatus, the image encoder, and the image decoder according to the oneor more embodiments may include a bus coupled to each unit in anapparatus as illustrated in FIGS. 1-2, 4-5, 9-10, and 14, and at leastone processor coupled to the bus. Also, each of the image encodingapparatus, the image decoding apparatus, the image encoder, and theimage decoder according to the one or more embodiments may include amemory coupled to the at least one processor that is coupled to the busso as to store commands, received messages or generated messages, and toexecute the commands.

While this invention has been particularly shown and described withreference to exemplary embodiments thereof, it will be understood bythose of ordinary skill in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the invention as defined by the appended claims. The exemplaryembodiments should be considered in a descriptive sense only and not forpurposes of limitation. Therefore, the scope of the invention is definednot by the detailed description of the invention but by the appendedclaims, and all differences within the scope will be construed as beingincluded in the present invention.

What is claimed is:
 1. An image decoding apparatus comprising: adeterminer which determines coding units having a hierarchical structurefor decoding an image, at least one prediction unit for predicting eachof the coding units, and at least one transformation unit for inverselytransforming each of the coding units, by using information about adivision shape of a coding unit, information about the at least oneprediction unit, and information about the at least one transformationunit, parsed from a received bitstream of an encoded video; and adecoder which parses from the bitstream transformation coefficientsgenerated by transformation according to the at least one transformationunit generated by dividing the coding unit, and reconstructs encodeddata of the at least one prediction unit by performing entropy decoding,inverse quantization, and inverse transformation on the parsedtransformation coefficients, and performs intra prediction or interprediction on the reconstructed encoded data and reconstructing theencoded video, wherein the coding units are split hierarchicallyaccording to a depth of the coding unit, and wherein the at least onetransformation unit comprises a transformation unit having a sizedifferent than a size of the least one prediction unit.
 2. The imagedecoding apparatus of claim 1, wherein the at least one prediction unitcomprises a plurality of prediction units, and wherein the at least onetransformation unit comprises a transformation unit having a larger sizethan the size of the plurality of prediction units.
 3. The imagedecoding apparatus of claim 1, wherein the size of the at least onetransformation unit is different from the size of the least oneprediction unit and a size of the coding unit.
 4. The image decodingapparatus of claim 1, wherein a video of the encoded video is encodedbased on information about a maximum size of a coding unit and a depthof the coding unit into which the coding unit is hierarchically splitinto the coding units of coded depths according to depths, the codingunit of a current depth is one of rectangular data units split from acoding unit of a upper depth, and wherein the coding unit of a currentdepth are split into coding units of a lower depth up to coding units ofa corresponding coded depth, independently from neighboring codingunits.